In recent years, image sensors such as CMOS sensors have evolved, and it has become possible to read out a captured image signal at high speed. As a result, the number of pixels of moving images such as full-HD moving images and 4K2K moving images has been increasing. Further, since it is also possible to obtain an oversampling effect by reading out signals of pixels in a manner such that the number of pixels for which readout is performed from an image sensor is greater than the number of recording pixels of a moving image and generating an image while thinning the signals during processing, it has become possible to generate high-definition moving images.
On the other hand, there is an upper limit to the number of pixels that can be processed so as to satisfy a frame rate requirement of a moving image, and if the upper limit is exceeded it is necessary to adopt a countermeasure in a development process. For example, when development is performed with respect to pixels that have been read out from an image sensor without changing the number of pixels and the image is not reduced until the final stage of the process, a heavy load is placed on the processing system because a large number of data items undergo processing. Therefore, a method can be considered which decreases a data amount to be processed by reducing an image at the earliest possible stage after performing a read-out operation from the image sensor, and subjecting the image after reduction to a development process.
For example, the following process is known as a process that performs thinning at the stage of a RAW image that has been obtained from an image sensor (Japanese Patent Laid-Open No. 2010-004472). According to the aforementioned process, a luminance signal and color signals are generated based on a RAW image; a filtering process is performed with respect to the luminance signal and the color signals, respectively; and thereafter pixels are thinned. The luminance signals and color signals that have been thinned are combined and returned to a RAW image. Because the process is configured in this manner, thinning processing can be implemented with respect to a RAW image. Further, a method has been disclosed that prevents aliasing due to thinning by changing filter coefficients when performing thinning in accordance with a reduction ratio.
However, the above described conventional methods do not give consideration to the fact that a band of a signal obtained from an image sensor differs depending on the color filter pattern and color signals of the sensor. There is thus a possibility that adverse effects of aliasing will arise when thinning processing is merely performed uniformly simply as a luminance signal.
For example, a case in which the pattern of a color filter of an image sensor is a Bayer pattern as shown in FIG. 6 will now be described. In the case of a filter pattern shown in FIG. 6, if G1 and G2 are regarded as the same G signal, the density of pixels with a G signal will be higher than the pixel density for each of the R and B signals, and a band of the G signal and bands of the R and B signals will be as shown in FIG. 7, respectively. The x-axis indicates the frequency space of an imaging subject in the horizontal (H) direction, the y-axis indicates the frequency space in the vertical (V) direction, and the spatial frequency increases with increasing distance from the origin point. A band of the G signal is within a white region (including a diagonal line region), and a region outside the white region represents an aliasing signal. In the white region, bands in the horizontal and vertical directions are equal to or less than a Nyquist frequency, and bands in oblique 45-degree and 135-degree directions are equal to or less than a frequency represented by 1/√2 of the Nyquist frequency. Further, bands of the R and B signals are within the diagonal line region, and a region outside the diagonal line region represents an aliasing signal. In the diagonal line region, bands in the horizontal and vertical directions are equal to or less than ½ of the Nyquist frequency, and bands in oblique 45-degree and 135-degree directions match with the white region. When an image that has been read out from the image sensor is reduced to a size that is equal to or less than ½ of the original size in the horizontal and vertical directions, since only signals within the bands (diagonal line region) remain for all of the color signals, there is almost no residual aliasing signal. On the other hand, when an image that has been read out from the image sensor is reduced to a size that is greater than or equal to ½ of the original size, for example, since bands within the broken line in the figure remain, with respect to the G signal, although diagonal aliasing (halftone dot region part) remains, almost no aliasing remains in the horizontal and vertical directions. However, with respect to the R and B signals, since an aliasing portion (the outside of the diagonal line region) already exists from before reducing is performed, an aliasing signal remains after reducing also.